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DUPLICATE 

1-00-144 P/10148 1 

EMBEDDING DATA IN MATERIAL 

The present invention relates to embedding data in material. Embodiments of 
the invention relate to watermarking material. 

In this application material means one or more of video material, audio 
5 material and data material. In this context, video is generic to still images and moving 
images. 
Steganographv 

Steganography is the embedding of data into material such as video material, 
audio material and data material in such a way that the data is imperceptible in the 
10 material. 

Data may be embedded as a watermark in material such as video material, 
audio material and data material. A watermark may be imperceptible or perceptible in 
the material. 

A watermark may be used for various purposes. It is known to use watermarks 
15 for the purpose of protecting the material against, or trace, infringement of the 

intellectual property rights of the owner(s) of the material. For example a watermark 

may identify the owner of the material. 

Watermarks may be "robust" in that they are difficult to remove from the 

material. Robust watermarks are useful to trace the provenance of material which is 
20 processed in some way either in an attempt to remove the mark or to effect legitimate 

processing such as video editing or compression for storage and/or transmission. 

Watermarks may be ''fragile" in that they are easily damaged by processing which is 

useful to detect attempts to remove the mark or process the material. 

Visible watermarks are useful to allow e.g. a customer to view an image e,g. 
25 over the Internet to determine whether they wish to buy it but without allowing the 

customer access to the unmarked image they would buy. The watermark degrades the 

image and the mark is preferably not removable by the customer. Visible watermarks 

are also used to determine the provenance of the material into which they are 

embedded. 
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It is known to embed a watermark into material by applying a spatial frequency 
transform to the material embedding the watermark in the spatial frequency transform 
and applying an inverse transform to the watermarked material. A scaling factor is 
applied to the watermark. It is desirable to choose a scaling factor to improve the 
ability of the watermark to withstand unauthorised attempts to remove it; allow 
efficient authorised removal; reduce degradation of the unmarked material; and ensure 
that the mark is imperceptible where an imperceptible mark is desired. Those 
properties may be incompatible. 

According to one aspect of the present invention, there is provided a method of 
embedding data in material, the method comprising the steps of: 

producing transform coefficients Ci representing a spatial frequency transform 
of the material, and 

combining the coefficients Ci with bits Ri of the data to produce modified 
coefficients Ci' where 
Ci'=Ci + cci Ri 

the method further comprising determining cci for each unmodified coefficient 
Ci as a function F{Cn}j of a predetermined set {Cn}i of transform coefficients Cn 
which set excludes the coefficient Ci wherein the coefficients are serially ordered and 
the coefficients Cn are coefficients preceding coefficient Ci. 

Preferably, the set {Cn}i of transform coefficients is: 

a) a set consisting of unmodified coefficients; or 

b) a set consisting of modified coefficients; or 

c) a set comprising modified and unmodified coefficients. 

Thus ai is adapted to each coefficient to which it is applied, allowing it to 
minimise degradation of the material. That also allows ai to make the embedded data 
more robust against processing which intentionally or unintentionally damages the 
embedded data. 

The set {Cn}j of coefficients used to calculate ai associated with coefficient Ci 
excludes Ci. As will become apparent from the method of removing the data Ri, that 
allows exact recalculation of ai in the removal process and thus exact removal of Ri to 
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restore the original material if no processing has occurred, and no clipping of the 
image in the spatial domain has occurred. 

The invention allows cci to be related to the other coefficients from which it is 
calculated by any suitable function. 
5 The transform may produce coefficients Ci in a plurality of frequency bands. 

The transform coefficients forming the set {Cn}i may be all in the same band. The 
transform coefficients forming the set {Cn}i may be in a plurality of bands. Using a set 
of coefficients {Cn}i in a plurality of bands allows the data Ri to be concealed in the 
material using material properties in bands other than the band containing the data Ri. 

10 In a preferred embodiment, the coefficients are serially ordered and the 

coefficients Cn are unmodified coefficients preceding coefficient Ci. During removal 
of the embedded data such ordering allows the coefficients to be used to calculate otji 
for a subsequent coefficient Cj. 

In such circumstances, the set {Cn}i may be: 

1 5 a) the set consisting of unmodified coefficients; or 

b) a set consisting of modified coefficients; or 

c) a set comprising modified and unmodified coefficients. 

According to another aspect of the present invention there is provided a method 
of removing data embedded in material according to the method of said one aspect, the 
20 method comprising the steps of: 

determining the values of bits Ri of the data; 
, calculating, for each modified coefficient Ci', the value of the said function 
F{Cn}i of the corresponding set {Cn}j of coefficients Cn to determine ai; and 

for each modified coefficient Ci', subtracting therefrom ai.Ri to restore the 
25 unmodified coefficient value Ci, wherein the coefficients are serially ordered and the 
coefficients Cn are coefficients preceding coefficient Ci. 

In a preferred embodiment, ai is calculated from a set {Cn}i of unmodified 
coefficients. The method thus uses the restored coefficient Ci as an unmodified 
coefficient Cn of another set {Cn}j of unmodified coefficients for restoring another 
30 coefficient Cj'. It will be appreciated that the set {Cn}j excludes the coefficient Ci'. 
The set {Cn}i is of unmodified coefficients allowing ai to be calculated exactly from 
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the material in which the data Ri is embedded. As a modified coefficient Ci' is 
restored to its original value it is then available to be used to calculate aj for another 
coefficient CjV 

In a preferred embodiment, the coefficients are serially ordered and the 
coefficients Cn are unmodified coefficients preceding coefficient Ci. During removal 
of the embedded data such ordering allows the coefficients to be used to calculate cci 
for a subsequent coefficient j 

In such circumstances, the set {Cn}i may be: 

a) the set consisting of unmodified coefficients; or 

b) a set consisting of modified coefficients; or 

c) a set comprising modified and unmodified coefficients. 

These and other aspects of the invention are specified in the claims to which 
attention is invited. 

For a better understanding of the present invention, reference will now be made 
by way of example to the accompanying drawings in which: 

Figure 1 is a schematic block diagram of a watermark embedding and removal 

system; 

Figure 2 is a more detailed schematic block diagram of an embedder of the 
system of Figure 1 ; 

Figures 3A and B illustrate an example of a window of coefficients and how 
the window relates to a coefficient Ci being modified to embed a bit of a watermark; 

Figure 4 is a flow diagram of a method of calculating strength a in accordance 
with an example of the invention; 

Figure 5 is a schematic block diagram of a watermark decoder; 

Figure 6 is a schematic block diagram of a watermark remover; 

Figure 7 is a flow diagram of a method of calculating strength a in accordance 
with an example of the invention; 

Figure 8 is a schematic diagram of an alternative, illustrative, set of coefficients 
usable to calculate a; 

Figure 9 is a schematic diagram illustrating the operation of frame stores in the 
embedder of Figure 2 and the remover of Figure 6; 
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Figures 10 and 11 are diagrams used herein below to describe wavelets 
transforms; and 

Figures 12 and 13 are diagrams of data structures of UMIDs. 
Overview 

5 Figure 1 illustrates a watermarking system, generally 10, for embedding, 

recovering and removing a watermark onto or from a video image I. The 
watermarking system 10 comprises a source 1 10 of the image I, a strength adapter 180. 
a watermark embedder 120, a watermark decoder 140, a watermark remover 130 and a 
store 150. The decoder and remover may be coupled to the embedder via a channel 

10 125 which may include a video processor, and/or a store. 

In overview, the watermark embedder 120 embeds a watermark onto a video 
image 1 15 to form a watermarked image F, the watermark decoder 140 recovers the 
watermark from the watermarked image V and the watermark remover 130 removes 
the watermark from the watermarked image F to produce a restored image I". The 

15 restored image I" may not equal the original image I, especially if the channel 125 
includes a processor and/or if clipping of the image in the spatial domain occurs. 

The watermark embedder 120 receives, in this example, as watermark data, a 
UMID. UMIDs are described in the section UMIDs below. The strength adapter 180 
determines the magnitude a of the watermark in relation to the video image 1^ the 

20 strength-being determined such that the watermark may be recovered whilst 
minimising its perceptibility to a viewer of the watermarked image I\ The 
watermarked image F may then be stored, and/or transmitted and/or routed for further 
processing, in the channel 125. 

The watermark decoder 140 generates a restored UMID 145 from the 

25 watermarked image F. The watermark remover 130 generates a restored image I" from 
the watermarked image F using the restored UMID. 
Watermark embedder. Figure 2. 

Figure 2 illustrates the watermark embedder 120 in more detail. The 
watermark embedder 120 comprises pseudo-random sequence generator 220, an error 

30 correction coding generator 200, a wavelet transformer 210, an inverse wavelet 
transformer 250, a first combiner 230, a data converter 225 and a second combiner 
240. The wavelet transformer 210 includes a frame store FS1. The inverse 
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transformer 250 includes a frame store FS2. The frame store FS1 stores a frame of 
unmodified coefficients Ci. The frame store FS2 stores a frame of modified 
coefficients Ci'. 

The error correction coding generator 200 receives the UMID and outputs an 
5 error correction coded UMID to the first combiner 230. The pseudo-random sequence 
generator 220 outputs a pseudo-random binary sequence (PRBS) Pi, where i is the i th 
bit of the sequence, to the first combiner 230. The PRBS has a length L x J of bits 
where J is the number of bits in the error correction encoded UMID. Each bit j of the 
error correction encoded UMID then modulates a section of length L of the PRBS. The 
10 first combiner 230 logically combines the error correction encoded UMID with the 
PRBS to produce a watermark having bits Ri. A bit Wj=0 of the error correction 
encoded UMID inverts L bits of the PRBS. A bit Wj=l of the error correction 
encoded UMID does not invert the PRBS. Thus bits Wj of the error correction encoded 
UMID are spread over L bits of the PRBS. The data converter 225 converts binary 1 to 
15 symbol +1 and binary 0 to symbol -1 to ensure that binary 0 bits contribute to a 
correlation value used in the decoder of Figure 5. 

The wavelet transformer 210 receives the video image I from the source 110 
and outputs wavelet coefficients to the second combiner 240. Wavelets are briefly 
discussed in the section Wavelets below. 
20 The second combiner 240 receives the watermark Ri, the wavelet coefficients 

Ci and watermark strength ori and outputs modified coefficients Ci' where 

Ci'=Ci + cciRi 

The inverse wavelet transformer 250 receives the modified coefficients Ci' and 
outputs a spatial domain watermarked image I\ 

25 The embedder includes an ECC generator 200. The use of error correction 

coding to produce an error correction coded UMID is advantageous since it allows the 
UMID 175 to be reconstructed more readily should some information be lost. This 
provides a degree of robustness to future processing or attacks against the watermark. 
The use of a pseudo-random sequence Pi to generate a spread spectrum signal for use 

30 as a watermark is advantageous since it allows the error correction coded UMID 205 
to be spread across a large number of bits. Also, it allows the watermark to be more 
effectively hidden and reduces the visibility of the watermark. Applying the 
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watermark to a wavelet transform of the image is advantageous since this reduces the 
perceptibility of the watermark. Furthermore, the strength of the watermark is 
adjusted by ai to ensure that the watermark is not perceptible. 

The operation of the error correction code generator 200 will now be described. 
5 The error correction code generator 200 receives a UMID. Typically the UMID will 
be a binary sequence of 3 1 bytes. The error correction code generator 200 typically 
outputs a 511 bit error correction coded binary sequence. Various error correction 
coding schemes are known. One approach uses BCH coding which corrects up to 31 
bit errors. The error correction rates can be further improved by using knowledge of 
10 the UMID format to help correct errors. One such approach is to check for invalid 
dates times GPS locations etc. 

The watermark is preferably embedded in predetermined regions of the wavelet 
transformed image. Most preferably the upper horizontal (hH,lV)and upper vertical 
(lH,hV) bands are used. These bands are chosen as watermarks embedded in these 
1 5 regions are not readily perceptible. The length of the pseudo-randomi sequence may be 
chosen such that the watermark fills the predetermined regions in each wavelet image. 
The regions in which the watermark is embedded may be within a border of 
unmodified coefficients thereby allowing the image to the spatially shifted without the 
watermark being lost. 
20 Calculating a. Figures 3 and 4. 

In accordance with an illustrative embodiment of the invention, for each 
coefficient Ci, a value of a, ai is calculated, ai is calculated as 

ai = F{Cn} i5 

where {Cn}j is a set of unmodified wavelet coefficients excluding Ci, which set may 
25 vary with i, that is respective values of ai are functions F of respective sets {Cn}j. This 
is shown as step S8 in Figure 4. 

The coefficients {Cn}i of the set may be in the same wavelet band as Ci or may 
be in different bands from Ci and from each other as described below with reference to 
Figure 8. 
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If the coefficients are in the same band as Ci, they are preferably in a window 
adjacent Ci. For example the set comprises N coefficients Ci- 1 to Ci-N as shown in 
Figure 3 and the embodiment will be described in the following with reference to that. 

The number N of coefficients may vary with Ci; thus for generality N is 
denoted as Ni. 

The function F may be any suitable function. In this illustrative embodiment F 
is such that 

ai = F{Cn}/ = — JY C: for n = i-1 to i-N for Ni*0 and ai = k for Ni = 0. 



Figure 3 A is a map of wavelet coefficients in a frame store 300, the coefficients 
being in level 1 of a wavelet transform. In a preferred embodiment, the coefficients Ci 
are modified only in the upper horizontal hH, IV and upper vertical 1H, hV bands to 
embed the watermark. However, coefficients in other bands and/or in other levels may 
be modified to embed a watermark. In the following only band hH, IV is considered. 

The wavelet coefficients are stored in the frame store 300 (also denoted FS1 in 
Figure 2) and in this example are stored as shown in Figure 3 A grouped in the bands. 
The coefficients are serially ordered. For example they may be serially ordered by a 
raster scan thereof. Other scanning patterns are known. Assuming serial ordering of 
the coefficients in each band, for each coefficient Ci to be modified, there is defined a 
set {Cn}j (herein also referred to as a 'window') of Ni coefficients excluding Ci. The 
set {Cn}j consists of the Ni coefficients Ci-1 to Ci-Ni preceding coefficient Ci on the 
same line , up to a maximum of for example M most recent coefficients. It will be 
noted that in the band hH, IV coefficient C 1 has no preceding coefficients, C2 has only 
one preceding coefficient, and so on. For coefficient CI ai is set to a predetermined 
value K. For subsequent coefficients the set comprises the totality of preceding 
coefficients. 

Thus ai is defined individually for each coefficient Ci to be modified. In the 
example above it is defined by the set of Ni unmodified coefficients preceding Ci. By 
choice of the appropriate function F, ai is adapted to the image such that image 
degradation can be minimised. In addition as will be discussed below in the section 
Remover, this allows ai to be recalculated from the watermarked image coefficients, 
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after those have been restored to their original values. This improves the accuracy of 
restoring the original image. 

Referring to Figure 4 the illustrative procedure for calculating Ci 5 is as 
follows:- 

The calculation procedure starts at step S2. At step S4, i is initialised with value 0. At 
step S6, i is incremented by 1 to calculate al at step S8 for coefficient Cl\ At step 
S10 the value of modified coefficient CI' is calculated. The procedure then reverts to 
step S6 and i is incremented. The procedure continues until all coefficients have been 
modified. 

In addition, the calculation of cri may be modified in one or both of the 
following ways:- 

1 ) If (xKcctl, it is incremented to <xtl, where ajL is a lower threshold; and 
if cci> ccth it is reduced to cc T h ? where ocjh is an upper threshold. 

2) The magnitude I C n I of each coefficient is compared with a threshold Cth- 
If I C n I >Cth then Cn is not included in the calculation of cci,; or 

if I C n | >C TH , then C n is clipped to (C„/ |C n |)C T H 

Watermark decoder and remover. Figures 5 and 6. 
Decoder Figure 5 

The operation of the watermark decoder 140 will now be explained in more 
detail with reference to Figure 5. The watermark decoder 140 receives the 
watermarked image F and outputs the restored UMID. The watermark decoder 140 
comprises a wavelet transformer 310, a reference pseudo-random sequence (PRBS) 
generator 320, a correlator 330, a selector 340 and a error correction coding decoder 
350. The PRBS generated by the generator 320 is identical to that generated by the 
PRBS generator 220 of Figure 2 and converted by a data converter (not shown ) to 
values +1 and -1 as described above. 

The wavelet transformer 310 receives the watermarked image F and, in known 
mariner, outputs the modified wavelet coefficients Ci'. The correlator 330 receives the 
reference pseudo-random sequence PRBS having symbols Pi of values +1 and -1 from 
the pseudo-random sequence generator 320, and the wavelet coefficients Ci' and 
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outputs a watermark image bit correlation sequence 335. The watermarked image bit 
correlation sequence is determined in the following way. 

The modified wavelet coefficients Ci'=Ci+aiRj where Rj are bits of PRBS 
modulated by error-correction encoded bits Wj of UMID. In the example given above 
5 there are 5 1 1 bits Wj. Each bit Wj modulates L bits of PRBS. There are JL bits in the 
modulated PRBS. 

For each error correction encoded bit Wj, the correlater 330 calculates a 
correlation value 

jL+L 

10 where j = 0, 1, 2... T-l, and T is the number of error correction encoded bits. In 

this example T=51 1. A sequence 335 of correlation values Sj is produced. 

The correlation sequence 335 is received by the selector 340 which outputs an 
uncorrected UMID 345. The selector 340 outputs a bit value "1" for a value of S' 
greater than 0 and a bit value "0" for S' less than or equal to 0. The error correction 
15 code decoder 350 receives the uncorrected UMID 345 and in known manner outputs 
the restored UMID 145. 

The reference PRBS Pi is synchronised with the modulated PRBS in the 
watermarked image. For that purpose a synchroniser (not shown) is used. Such 
synchronisation is known in the art. 
20 Remover Figure 6. 

The watermark remover 130 receives the restored UMID 145, and the 
watermarked image I' and outputs a restored image I". The watermark remover 130 
comprises a pseudo-random sequence generator 420 for generating a reference pseudo- 
random sequence Pi identical to that produced by generators 220 and 320, a spread 
25 spectrum signal generator 430 which produces, via a data converter 425, a restored 
watermark Rp having bit values +1 and -1 from the restored UMID 145 and the 
pseudo-random sequence Pi. The reference sequence Pi is synchronised with the 
modulated sequence in the watermarked image in known manner. 

The watermark remover 130 further comprises a wavelet transformer 410 
30 which produces modified wavelet coefficients CP from the watermarked image P, a 



1-00-144 P/10148 



11 



strength estimator 460 for calculating ai and a combiner 440 which calculates restored 
wavelet coefficient values according to the equation 

Ci = Ci' - ai. Ri\ 

The restored wavelet coefficients C are fed to an inverse wavelet transformer 
450 which outputs the restored image I". 

Calculating ai. Figure 7. 

In accordance with the illustrative embodiment of the invention, ai is 
calculated in the embedder as described above in the section Calculating a. The 
estimator 460 of the remover of Figure 6 recalculates a in analogous manner from 
coefficients Ci which have been restored to their original values. 

Thus referring for example to Figure 3A and to Figures 6 and 7, the modified 
coefficients Ci' are stored in a frame store 300 indicatd as FS3 in the wavelet 
transformer of Figure 6 in the same way as shown in figure 3A and they are serially 
ordered in the same way as described with reference to Figure 3A. It will be recalled 
that coefficient Ci' has no preceding coefficients so al = k and CI = CI*- kRl. For 
each subsequent coefficient Ci, ai is calculable from the set of Ni of preceding 
restored coefficients, all of which have been restored to their original value according 
to 

Ci = Ci' - Ri\ 

Referring to Figure 7, the calculation procedure starts at step S5. At step S7, i 
is initialised to 0. At step S9, i is incremented by 1 to calculate al at step Sll for 
coefficient CI'. At step SI 3 the original value CI is calculated from coefficients Cl\ 
The procedure then reverts to step S9 and i is incremented. The procedure continues 
until all coefficients Ci' have been restored to their original values Ci. 

As in the embedder of Figure 2, the calculation of a may be modified in one or 
both of the following ways:- 

1) If ai<a T L> it is incremented to a T L, where a T L is a lower threshold; and 
if ai> a T H it is reduced to a T H, where a T H is an upper threshold. 

2) The magnitude | Cn | of each coefficient is compared with a threshold Cth. 
If I Cn I >Cth then Cn is not included in the calculation of ai; or 
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if I Cn | >C T h, then C n is clipped to (Cn/|Cn|)C TH . 



Modifications. 

5 As mentioned above the coefficients from which the value of ori is 

calculated may be in different bands to the related coefficient Ci which is to be 
modified or restored to its original value. Thus by way of example, referring to Figure 
8, the set of coefficients {Cn}j used to calculate cci of band hH, IV may be in the other 
bands. In the example of Figure 8 the set {Cn}j is shown as including coefficients Cli, 

10 C2i and C3i which are at positions related to the position of coefficient Ci. In this 
way, image properties in other bands are taken into account in calculating cci to ensure 
that the watermark is imperceptible. 

The coefficients Cli, C2i and C3i used to modify or restore Ci, may be 
coefficient which are never modified. That can be done by modifying only 

1 5 coefficients in one or more bands such as hH, IV and leaving the coefficients in other 
bands unmodified. Alternatively at least some of the coefficients Cli, C2i and C3i 
used to modify or restore Ci may be modified. That can be done by storing the 
coefficients in a frame store 300 as shown in Figure 3 or 8 and by reading out 
coefficients in an order which allows the procedures of Figures 4 and 7 to be followed. 

20 It will be appreciated that whilst the foregoing discussion refers for ease of 

explanation to only 3 coefficients Cli, C2i and C3i in 3 bands in one level, in practice 
many, more coefficients may be used and the coefficients may be in more than three 
bands and in more than one level. 
Other transforms 

25 Whilst the invention has been described by way of example with reference to 

Wavelet transforms, it may be used with other transforms for example DCT. 
Other material 

Whilst the invention has been described by way of example with reference to 
material comprising video material ( still or moving images), it may be applied to other 
30 material, for example audio material and data material. 
PRBS 
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As described hereinabove, the PRBS has a length of L J where J is the number 
of bits in a UMID. Thus each bit Wj of the UMID modulates a section of length L of 
the PRBS. Instead, it may have a length of L bits and be repeated for each bit j of the 
UMID. 

5 

Other Watermark data . 

Whilst the invention has been described by way of example with reference to 
UMIDs as the watermark data, it may be used with other data as the watermark. 

Using modified coefficients to calculate ai 
10 The foregoing embodiment calculates ai using unmodified coefficients. In 

alternative embodiments a is calculated using modified coefficients or a combination 
of modified and unmodified coefficients. The coefficients Ci are serially ordered. The 
coefficients used to calculate ai for coefficient Ci are coefficients preceding i on the 
serial order. 

15 Referring to Figures 2, 6 and 9 frames stores FS1, FS2, FS3 and FS4 are 

provided in the wavelet transformer 210, the inverse wavelet transformer 250, the 
wavelet transformer 410 and the inverse wavelet transformer 450. Frame stores 
FSland FS4 store unmodified coefficients. Frame stores FS2 and FS3 store modified 
coefficients C ; i. 

20 Thus there are available both at the encoder and at the remover serially ordered 

sets of unmodified and modified coefficients. 

, In the embedder of Figure 2, as coefficients Ci in store FS1 are modified, they 
are stored in FS2 as coefficients C/'. Thus modified coefficients Ci' are available to 
calculate ai. Thus the set {Cn}z used to calculate ai for modifying coefficient Ci may 

25 comprise modified coefficients C preceding Ci optionally together with unmodified 
coefficients C preceding Ci. 

At the remover modified coefficients C/' are stored in store FS3. As the 
coefficients are restored, restored coefficients Ci are stored in store FS4. Thus 
modified coefficients C are available to calculate ai optionally together with restored 

30 coefficients C 



\ 
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As diagrammatically shown in Figure 9, sets of coefficients preceding a 
coefficient Ci or Cf are present in all four frame stores FS1, FS2, FS3 and FS4. 
Shape of sets (Cn}/ 

A set {Cn}/ may have any convenient shape. Where ai is calculated only from 
coefficients preceding Ci, the set may consist of coefficients immediately preceding 
Ci. Where the coefficients are raster scanned to serially order them, the set may 
consist of coefficients on the same scanning line as Ci. Alternatively, it may consist of 
coefficients on that line and a preceding line. Other shapes are possible. 
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Wavelets 

Wavelets are well known and are described in for example i; A Really 
Friendly Guide to Wavelets " by C Valens, 1999 and available at 
http://perso.wanadoo.fr/polyvalens/clemens/wavelets/wavelets.html. 

Valens shows that the discrete wavelet transform can be implemented as an 
iterated filter bank as used in sub-band coding, with scaling of the image by a factor of 
2 at each iteration. 

Thus referring to Figure 1 1, a spatial domain image is applied to a set of high 
pass HP and low pass LP filters. At level 1, the first stage of filtering, the image is 
filtered horizontally and vertically and, in each direction, scaled down by a factor of 2. 
In level 2, the low pass image from level 1 is filtered and scaled in the same way as in 
level 1 . The filtering and scaling may be repeated in subsequent levels 3 onwards. 

The result is shown schematically in Figure 10. Figure 10 is a representation 
normal in the art. At level one the image is spatially filtered into four bands: the lower 
horizontal and vertical band, lHi, IV! ; the upper horizontal band hHi, lVj; the upper 
vertical band lHi, hVi; and the upper horizontal and vertical band, hH u hVi. At level 
2, the lower horizontal and vertical band, lHj, IV i is filtered and scaled into the lower 
horizontal and vertical band, 1H 2 , 1V 2 ; the upper horizontal band hH 2 , 1V 2 ; the upper 
vertical band 1H 2 , hV 2 ; and the upper horizontal and vertical band, hH 2 , hV 2 . At level 3 
(not shown in Figure 10), the lower horizontal and vertical band, 1H 2 , 1V 2 is further 
filtered and scaled. 
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UMIDs 

The UMID or Unique Material Identifier is described in SMPTE 
Journal March 2000. Referring to Figure 1 1 an extended UMID is shown. It 
5 comprises a first set of 32 bytes of basic UMID and a second set of 32 bytes of 

signature metadata. 

The first set of 32 bytes is the basic UMID. The components are: 
•A 12-byte Universal Label to identify this as a SMPTE UMID. It defines the 
type of material which the UMID identifies and also defines the methods by which the 
1 0 globally unique Material and locally unique Instance numbers are created. 

•A 1-byte length value to define the length of the remaining part of the UMID. 
•A 3-byte Instance number which is used to distinguish between different 
'instances' of material with the same Material number. 

•A 16-byte Material number which is used to identify each clip. Each Material 
1 5 number is the same for related instances of the same material. 

The second set of 32 bytes of the signature metadata as a set of packed 
metadata items used to create an extended UMID. The extended UMID comprises the 
basic UMID followed immediately by signature metadata which comprises: 

•An 8-byte time/date code identifying the time and date of the Content Unit 
20 creation. 

•A 12-byte value which defines the spatial co-ordinates at the time of Content 
Unit creation. 

•3 groups of 4-byte codes which register the country, organisation and user 

codes 

25 Each component of the basic and extended UMIDs will now be defined in turn. 

The 12-byte Universal Label 



The first 12 bytes of the UMID provide identification of the UMID by the 
registered string value defined in table 1 . 



Byte No. 


Description 


Value (hex) 


1 


Object Identifier 


06h 


2 


Label size 


OCh 
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J 


Designation: ISO 


2Bh 


4 


Designation: SMPTE 


34h 


5 


Registry: Dictionaries 


Olh 


6 


Registry: Metadata Dictionaries 


Olh 


7 


Standard: Dictionary Number 


Olh 


8 


Version number 


Olh 


9 


Class: Identification and location 


Olh 


10 


Sub-class: Globally Unique Identifiers 


Olh 


11 


Type: UMID (Picture, Audio, Data, Group) 


01, 02,03, 04h 


12 


Type: Number creation method 


XXh 



Table 1 : Specification of the UMID Universal Label 

The hex values in table 1 may be changed: the values given are examples. 
Also the bytes 1-12 may have designations other than those shown by way of example 
5 in the table. Referring to the Table 1 , in the example shown byte 4 indicates that bytes 
5-12 relate to a data format agreed by SMPTE. Byte 5 indicates that bytes 6 to 10 
relate to "dictionary" data. Byte 6 indicates that such data is "metadata" defined by 
bytes 7 to 10. Byte 7 indicates the part of the dictionary containing metadata defined 
by bytes 9 and 10. Byte 10 indicates the version of the dictionary. Byte 9 indicates 
1 0 the class of data and Byte 1 0 indicates a particular item in the class. 

In the present embodiment bytes 1 to 10 have fixed preassigned values. Byte 
1 1 is variable. Thus referring to Figure 12, and to Table 1 above, it will be noted that 
the bytes 1 to 10 of the label of the UMID are fixed. Therefore as shown in Figure 13 
they may be replaced by a 1 byte 'Type' code T representing the bytes 1 to 10. The 
1 5 type code T is followed by a length code L. That is followed by 2 bytes, one of which 
is byte 1 1 of Table 1 and the other of which is byte 12 of Table 1, an instance number 
(3 bytes) and a material number (16 bytes). Optionally the material number may be 
followed by the signature metadata of the extended UMID and/or other metadata. 

The UMID type (byte 1 1) has 4 separate values to identify each of 4 different 
20 data types as follows: 

*01h* = UMID for Picture material 
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4 02h' = UMID for Audio material 
'03h' = UMID for Data material 

c 04h' = UMID for Group material (i.e. a combination of related essence). 

The last (12th) byte of the 12 byte label identifies the methods by which the 
5 material and instance numbers are created. This byte is divided into top and bottom 
nibbles where the top nibble defines the method of Material number creation and the 
bottom nibble defines the method of Instance number creation. 

Length 

The Length is a 1-byte number with the value '13h' for basic UMIDs and '33h' 
1 0 for extended UMIDs, 

Instance Number 

The Instance number is a unique 3 -byte number which is created by one of 
several means defined by the standard. It provides the link between a particular 
'instance' of a clip and externally associated metadata. Without this instance number, 
1 5 all material could be linked to any instance of the material and its associated metadata. 

The creation of a new clip requires the creation of a new Material number 
together with a zero Instance number. Therefore, a non-zero Instance number 
indicates that the associated clip is not the source material. An Instance number is 
primarily used to identify associated metadata related to any particular instance of a 
20 clip. 

Material Number 

The 16-byte Material number is a non-zero number created by one of several 
means identified in the standard. The number is dependent on a 6-byte registered port 
ID number, time and a random number generator. 
25 Signature Metadata 

Any component from the signature metadata may be null-filled where no 
meaningful value can be entered. Any null-filled component is wholly null-filled to 
clearly indicate a downstream decoder that the component is not valid. 

The Time-Date Format 
30 The date-time format is 8 bytes where the first 4 bytes are a UTC (Universal 

Time Code) based time component. The time is defined either by an AES3 32-bit 
audio sample clock or SMPTE 12M depending on the essence type. 
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The second 4 bytes define the date based on the Modified Julian Data (MJD) as 
defined in SMPTE 309M. This counts up to 999,999 days after midnight on the 17th 
November 1858 and allows dates to the year 4597. 

The Spatial Co-ordinate Format 
5 The spatial co-ordinate value consists of three components defined as follows: 

•Altitude: 8 decimal numbers specifying up to 99,999,999 metres. 

•Longitude: 8 decimal numbers specifying East/West 180.00000 degrees (5 
decimal places active). 

•Latitude: 8 decimal numbers specifying North/South 90.00000 degrees (5 
1 0 decimal places active). 

The Altitude value is expressed as a value in metres from the centre of the earth 
thus allowing altitudes below the sea level. 

It should be noted that although spatial co-ordinates are static for most clips, 
this is not true for all cases. Material captured from a moving source such as a camera 
1 5 mounted on a vehicle may show changing spatial co-ordinate values. 

Country Code 

The Country code is an abbreviated 4-byte alpha-numeric string according to 
the set defined in ISO 3 166. Countries which are not registered can obtain a registered 
alpha-numeric string from the SMPTE Registration Authority. 
20 Organisation Code 

The Organisation code is an abbreviated 4-byte alpha-numeric string registered 
with SMPTE. Organisation codes have meaning only in relation to their registered 
Country code so that Organisation codes can have the same value in different 
countries. 

25 

User Code 

The User code is a 4-byte alpha-numeric string assigned locally by each 
organisation and is not globally registered. User codes are defined in relation to their 
registered Organisation and Country codes so that User codes may have the same 
30 value in different organisations and countries. 
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CLAIMS 

1. A method of embedding data in material, the method comprising the 
steps of: 

producing transform coefficients Ci representing a transform of the material, 

5 and 

combining the coefficients Ci with data symbols Ri to produce modified 
coefficients Ci' where 
Ci'=Ci + oci Ri 

the method further comprising determining oci for each unmodified coefficient 
10 Ci as a function F{Cn}i of a predetermined set {Cn}i of transform coefficients Cn 
which set excludes the coefficient Ci wherein the coefficients are serially ordered and 
the coefficients Cn are coefficients preceding coefficient Ci. 

2. A method according to claim 1 wherein the coefficients of the set {Cn}j 
1 5 vary with i. 

3. r A method according to claim 1 or 2, wherein the number Ni of 
coefficients in the set {Cn}i varies with i. 

20 4. A method according to claim 1, 2 or 3, wherein the coefficients of the 

set {Cn}s have a predetermined positional relationship with the coefficient Ci to be 
modified. 

5. A method according to claim 1, 2, 3 or 4, wherein the coefficients 
25 represent a spatial frequency transform of the material. 

6. A method according to claim 1, 2, 3 or 4, wherein the coefficients 
represent a wavelet transform of the material. 

30 7. A method according to claim 6, wherein the transform produces 

coefficients Ci in a plurality of bands. 
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8. A method according to claim 7, wherein the transform coefficients 
forming the set {Cn}j are all in the same band. 

5 9. A method according to claim 7, wherein the transform coefficients 

forming the set {Cn}i are in a plurality of bands. 

10. A method according to any preceding claim wherein the said function 
F{Cn}i is such that 

10 ai = ~^^Y, C n for n = f - J t0 i-Ni f° r N <*° and ai = k for Nj = 0 

where Ni is the number of coefficients Cn in set i. 

11. A method according to any preceding claim, wherein the said data 
15 symbols Ri are of a pseudo random symbol sequence having symbols Pi modulated by 

data Wj to be embedded. 

12. Apparatus for embedding data in material, comprising 

a transformer for producing transform coefficients Ci representing a transform 
20 of the material, and 

a combiner for combining the coefficients Ci with data symbols Ri to produce 
modified coefficients Ci' where 
Ci'=Ci + ai Ri 

the apparatus further comprising 
25 a calculator for calculating ai for each unmodified coefficient Ci as a function 

F{Cn}j of a predetermined set {Cn}j of transform coefficients Cn which set excludes 
the coefficient Ci, wherein the coefficients are serially ordered and the coefficients Cn 
are coefficients preceding coefficient Ci. 

30 13. Apparatus according to claim 12, wherein the coefficients of the set 

{Cn}- vary with i. 
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14. Apparatus according to claim 12, or 13, wherein the unmodified 
coefficients of the set {Cn}j have a predetermined positional relationship with the 
coefficient Ci to be modified. 

5 

15. Apparatus according to claim 12, 13 or 14, wherein the coefficients 
represent a spatial frequency transform of the material. 

16. Apparatus according to claim 12, 13, or 14, wherein the coefficients 
1 0 represent a wavelet transform of the material 

17. Apparatus according to claim 16, wherein the transformer produces 
coefficients Ci in a plurality of frequency bands. 

15 18. Apparatus according to claim 17, wherein the transform coefficients 

forming the set {Cn}j are all in the same band. 

19. Apparatus according to claim 18, wherein the transform coefficients 
forming the set {Cn}j are in a plurality of bands. 

20 

20. Apparatus according to any one of claims 12 to 19, wherein the said 
function F{Cn}i is such that 

ai = — -41l c I for n = *'l t0 l ' N i for N i^° and ai = k for Ni = 0 
where Ni is the number of coefficients Cn in set i. 

25 

21. A method or apparatus according to any preceding claim, wherein the 
data is imperceptibly embedded in the other material. 

22. A method or apparatus according to any preceding claim, wherein the 
30 set {Cn}i consists of unmodified coefficients. 
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23. A method or apparatus according to any one of claims 1 to 21, wherein 
the set {Cn}i consists of modified coefficients preceding Ci where the coefficients are 
serially ordered. 

24. A method or apparatus according to any one of claims 1 to 21, wherein 
the set {Cn}j comprises at least one modified coefficient and at least one unmodified 
coefficient. 

25. A method of removing data embedded in material according to the 
method of any one of claims 1 to 12, the detecting method comprising: 

determining the values of the data symbols Ri; 

calculating, for each modified coefficient Ci', the value of the said function 
F{Cn}i of the corresponding set {Cn}j of coefficients Cn to determine cci; and 

for each modified coefficient Ci', subtracting therefrom cci.Ri to restore the 
unmodified coefficient value Ci, wherein the coefficients are serially ordered and the 
said set {Cn}i consists of modified coefficients preceding coefficient Ci. 

26. A method according to claim 25, wherein the said set {Cn}i consists of 
restored coefficients Ci and comprising the further step of using a restored coefficient 
Ci as a coefficient of another set {Cn}j of coefficients for restoring another coefficient 
Cj. 

27. Apparatus according to claim 25 or 26, wherein the said set {Cn}i 
comprises at least one modified coefficient and at least one restored coefficient, the 
coefficients preceding Ci. 

28. A method according to any one of claims 25 to 27, wherein the step of 
determining the values of the data bits Wj embedded in material according to the 
method of claim 11, comprises correlating a reference pseudo random symbol 
sequence with the modified coefficients Ci' and decoding the correlation values to 
determine the data Wj modulating the pseudo random sequence and remodulating the 
reference sequence with the said data to restore Ri. 
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29. Apparatus for removing data embedded in material according to the 
method of any one of claims 1 to 1 1, the apparatus comprising: 

a processor for determining the values of the symbols Ri; 
5 a calculator for calculating, for each modified coefficient Ci\ the value of the 

said function F{Cn}j of the corresponding set {Cn}i of coefficients Cn to determine ai; 
and 

a subtractor which, for each modified coefficient Ci\ subtracts therefrom ori.Ri 
to restore the unmodified coefficient value Ci, which thereby becomes available for 
10 use as an unmodified coefficient of another set (Cn}j of unmodified coefficients Cn 
for restoring another coefficient Ci', wherein the coefficients are serially ordered and 
the said set {Cn}i consists of coefficients preceding coefficient Ci. 

30. Apparatus according to claim 29, wherein the said set {Cn}i consists of 
15 restored coefficients Ci and comprising the further step of using a restored coefficient 

Ci as a coefficient of another set {Cn}i+1 of coefficients for restoring another 
coefficient Ci+1. 

31. Apparatus according to claim 30, wherein the said set {Cn}i consists of 
20 modified coefficients preceding coefficient Ci. 

. 32. Apparatus according to claim 30, wherein the said set {Cn}i comprises 
at least one modified coefficient and at least one restored coefficient, the coefficients 
preceding Ci. 

25 

33. Apparatus according to claim 29, 30, 31 or 32, wherein the means for 
determining the values of the data bits Wj embedded in the material according to the 
method of claim 12, comprises a correlator for correlating a reference pseudo random 
symbol sequence with the modified coefficients Ci\ a decoder for decoding the 
30 correlations to determine the data Wj modulating the modulated sequence and a 
modulator for remodulating the reference sequence with the said data to restore Ri. 
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34. A computer program product arranged to carry out the method of any 
one of claims 1 to 1 1 when run on a computer. 

35. A computer program product arranged to carry out the method of any 
one of claims 25 to 30 when run on a computer. 

36. A method or apparatus according to any preceding claim, wherein the 
material is video material. 

37. A method or apparatus according to any one of claims 1 to 35, wherein 
the material is audio material. 

38. A method or apparatus according to any one of claims 1 to 35, wherein 
the material is audio/visual material. 

39. A method of embedding data in material substantially as herein before 
described with reference to the accompanying drawings. 

40. Apparatus for embedding data in material substantially as herein before 
described with reference to the accompanying drawings. 

41. A method of removing data embedded in material substantially as 
herein before described with reference to the accompanying drawings. 

42. Apparatus for removing data embedded in material substantially as 
herein before described with reference to the accompanying drawings. 
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ABSTRACT 



EMBEDDING DATA IN MATERIAL 

A method of embedding data in material, comprises the steps of: 
producing transform coefficients Ci representing a spatial frequency transform 
of the material, and 

combining the coefficients Ci with the data bits Ri to produce a modified 
coefficient Ci' where 
Ci'=Ci + cti Ri 

the method further comprising determining oti for each unmodified coefficient 
Ci as a function F{Cn},- of a predetermined set {Cn}j of transform coefficients Cn 
which set excludes the coefficient Ci. 

Preferably, the set {Cn}i of transform coefficients is: 

a) a set consisting of unmodified coefficients; 

b) a set consisting of modified coefficients; or 

c) a set comprising modified and unmodified coefficients. 

A corresponding embedding apparatus and corresponding data removal method 
and apparatus are also disclosed. 



[Figures 3A, B and 4] 
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